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Abstract 

In  the  summer  of  2003,  Dr.  William  Winkenwerder  charged  senior  leaders  of  the 
military  healthcare  system  to  develop  business  plans  to  share  capabilities  between  all 
branches  of  service.  The  San  Antonio  Multi-Service  Market  established  the  Consult  and 
Appointment  Management  Office  in  November  2004,  to  consolidate  resources  and 
optimize  healthcare  delivery.  The  purpose  of  this  analysis  was  to  develop  a  framework  to 
evaluate  the  impact  of  the  Consult  and  Appointment  Management  Office  (CAMO)  on  the 
delivery  of  healthcare  in  the  San  Antonio  Multi-Service  Market.  The  study  used 
Kissick’s  Iron  Triangle  as  the  framework  for  evaluation.  The  total  direct  cost  of  the 
CAMO  included  set  up  costs  of  $  1 .8  million  and  annual  operating  costs  of  $  1 .9  million. 
Descriptive  and  inferential  statistics  were  used  to  compare  changes  in  baseline  data  for 
quality  and  access  measures.  Patients  reported  decreased  satisfaction  with  the  ease  of 
making  appointments  by  phone  (F  =  6.881,  p  <  .01)  and  the  number  of  days  between 
making  an  appointment  and  being  seen  by  provider  (F=  8.852,  p  <  .05).  No  significant 
changes  were  identified  for  telephonic  access  standards.  Management  must  continue  to 
monitor  performance  metrics  to  evaluate  the  impact  of  the  Consult  and  Appointment 
Management  Office. 
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Disclaimer 

The  views  expressed  in  this  study  are  those  of  the  author  and  do  not  reflect  the 
official  policy  or  position  of  the  Department  of  the  Army,  Department  of  Defense, 
Brooke  Army  Medical  Center,  or  the  United  States  Government. 

Statement  of  Ethical  Conduct  in  Research 
The  author  declares  no  conflict  of  interest  or  financial  incentives  in  any  product  or 
service  mentioned  in  this  analysis.  The  confidentiality  of  individuals  whose  data  may 
have  been  used  in  this  study  was  protected  at  all  times  and  under  no  circumstances 
will  be  discussed  or  released  to  outside  agencies. 
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Introduction 

In  the  summer  of  2003,  Dr.  William  Winkenwerder  charged  senior  leaders  of  the 
military  healthcare  system  to  develop  business  plans  to  share  capabilities  between  all 
branches  of  service.  In  response  to  Dr.  Winkenwerder’ s  directives  and  the  TRICARE 
Governance  Plan  of  2003,  Brigadier  General  Green  then  commander  of  Wilford  Hall 
Medical  Center,  and  Brigadier  General  Fox  then  commander  of  Brooke  Army  Medical 
Center,  agreed  to  coordinate  Air  Force  and  Army  medical  resources.  The  agreement  led 
to  the  formation  of  the  San  Antonio  Multi  Service  Market  (SAMSM).  The  purpose  of  the 
San  Antonio  Multi  Service  Market  was  to  consolidate  resources,  eliminate  redundancy, 
and  optimize  the  use  of  the  military  healthcare  facilities  (San  Antonio  Multi-Market 
Office,  2003). 

In  order  to  support  the  objectives  of  the  SAMSM,  Brigadier  General  Green  and 
Brigadier  General  Fox  proposed  the  creation  of  the  Consult  and  Appointment 
Management  Office  (CAMO)  to  the  Surgeons  General  of  the  Air  Force  and  Army.  The 
Surgeons  General  directed  the  establishment  of  the  CAMO  on  July  19,  2004.  Prior  to  the 
CAMO,  appointment  management  was  handled  primarily  at  the  clinic  level.  Consults 
had  extremely  limited  administrative  oversight.  Wilford  Hall  Medical  Center  and  Brooke 
Army  Medical  Center  had  independently  established  systems  for  central  appointing. 
However,  the  majority  of  clinics  exercised  the  option  not  to  use  the  central  appointing 
systems.  Further,  the  systems  lacked  the  ability  to  coordinate  medical  resources  in  the 
multi-service  market  and  provided  inadequate  management  of  consults  and  referrals.  The 
CAMO  addressed  these  deficiencies  by  consolidating  operations  into  one  central 
organization  with  the  ability  to  view  appointment  templates  and  schedule  patients  at  all 
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military  treatment  facilities  in  the  local  market.  Two  departments  of  the  CAMO  were 
dedicated  to  actively  manage  consults  and  referrals. 

Conditions  that  Prompted  the  Study 

The  CAMO  is  a  unique  organization  in  the  military  healthcare  system.  Although 
other  multi  service  markets  are  considering  implementation  of  similar  operations,  no 
other  centralized  appointment  and  consult  management  service  of  this  scope  exists  in  the 
military  healthcare  system  (S.  Graham,  personal  communication.  May  25,  2006). 
Performance  measures  are  closely  monitored  internally  and  by  the  SAMSM.  However, 
no  formal  studies  have  examined  the  impact  of  the  CAMO  in  terms  of  value  added  to  the 
delivery  of  healthcare.  Advances  in  information  management  systems  and  call 
distribution  systems  have  facilitated  the  collection  and  processing  of  data.  Post¬ 
implementation  data  are  now  available  to  analyze  the  performance  of  the  CAMO. 

Statement  of  the  Problem 

The  problem  statement  for  this  study  is  to  determine  the  impact  of  the  Consult  and 
Appointment  Management  Office  (CAMO)  on  health  care  delivery  in  the  San  Antonio 
Multi  Service  Market  since  implementation  in  November  2004.  The  specific  objectives 
for  this  analysis  are  listed  below: 

1 .  Review  the  mission  and  objectives  of  the  CAMO 

2.  Review  literature  pertaining  to  consult  and  referral  management,  central 
appointing,  and  aspects  of  patient  satisfaction  related  to  appointments  and  consults. 

3.  Define  and  observe  the  standard  operating  procedures  for  the  CAMO. 
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4.  Identify  key  assumptions  and  limitations  of  the  analysis. 

5.  Gather  and  analyze  data  related  to  referral  management,  telephonic  access 
standards,  and  patient  satisfaction. 

6.  Determine  if  there  was  a  significant  change  in  referral  management, 
telephonic  access  standards,  and  patient  satisfaction  after  implementation  of  the  CAMO. 

7.  Report  findings  and  prepare  recommendations  for  further  evaluation. 

Literature  Review 

Mission  of  the  Consult  and  Appointment  Management  Office 

The  SAMSM  serves  over  212,000  eligible  beneficiaries  and  provides  healthcare 
for  more  than  1.3  million  outpatient  encounters  at  military  treatment  facilities,  annually. 
The  workload  within  the  SAMSM  generates  1 .6  million  relative  value  units  (RVU)  and 
34,500  relative  weighted  products  (RWP)  each  year.  The  SAMSM  captures  88%  of  the 
outpatient  workload  and  96  percent  of  the  inpatient  workload  within  military  treatment 
facilities  (Cuda,  2006,  January).  The  CAMO  is  a  tremendous  asset  for  coordinating 
resources  in  the  direct  care  system. 

The  CAMO  receives  an  average  of  40,000  calls  each  month  and  schedules  more 
than  30,000  appointments  monthly.  Administrative  oversight  is  also  provided  for 
approximately  1 200  deferrals  to  the  network  on  a  monthly  basis.  The  CAMO  manages 
95  percent  of  specialty  care  consults  in  the  SAMSM,  scheduling  more  than  700  external 
consults  monthly.  Appointment  scheduling  services  are  provided  to  over  1 00  clinics  in 
the  local  market  (Cuda,  2006,  January).  The  CAMO  does  not  provide  scheduling  or 
reporting  services  for  infectious  disease  clinics  or  behavioral  health  clinics.  A  select  few 
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clinics  retained  the  right  to  schedule  appointments  internally,  including  BAMC 
Pulmonary  Medicine. 

The  mission  of  the  CAMO  is  three- fold.  The  office  serves  as  ( 1 )  a  central 
specialty,  referral,  and  primary  care  focal  point  for  all  health  care  appointing  in  the  San 
Antonio  Multi  Service  Market,  (2)  a  single  source  for  external  medical  treatment 
facilities  to  access  San  Antonio  Multi  Service  Market  health  care  resources,  and  (3)  a 
primary  reporting  mechanism  for  specialty  care  results  for  patients  seen  in  the  network 
(Cuda,  2006,  January).  The  primary  objective  of  the  CAMO  is  to  maximize  use  of  the 
direct  care  system.  Additionally,  the  CAMO  was  designed  to  recapture  network 
workload,  reduce  high-dollar  network  referrals,  and  improve  the  administrative  process 
for  referrals  (San  Antonio  Multi-Market  Office,  2004). 

Organizational  Structure  and  Operations 

Staffing,  resources,  and  funding  for  the  CAMO  are  split  between  the  Air  Force 
and  Army.  The  Memorandum  of  Agreement  between  Wilford  Hall  Medical  Center  and 
Brooke  Army  Medical  Center  (BAMC)  outline  the  responsibilities  of  the  Air  Force  and 
Army.  The  CAMO  employs  general  schedule  (GS)  employees  and  contractors  under 
both  services.  The  Air  Force  provides  60  percent  of  the  staffing  based  on  enrollment 
demographics  for  beneficiaries  of  the  SAMSM.  Combining  the  two  previously  existing, 
service-specific  organizations  into  one  functional  unit  presented  the  first  challenge. 
Managing  the  service-specific  personnel  systems  and  contracts  continues  to  present  daily 
challenges  to  the  five  administrators  of  the  CAMO. 
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The  CAMO  uses  a  vertical  organizational  structure  to  align  departments  by 
function.  Appendix  A  illustrates  the  organizational  structure.  The  structure  provides 
efficiencies  through  specialization,  especially  when  considering  the  lack  of 
standardization  for  appointing  procedures  among  the  clinics  and  facilities.  The  CAMO 
employs  63  full  time  employees  to  accomplish  the  mission.  In  addition  to  the  three 
departments  previously  discussed,  the  CAMO  also  has  a  department  designed  to  receive 
and  distribute  consults.  Eight  part-time  employees  provide  flexibility  to  manage  peak 
call  volume  periods. 

Internal  consults  and  external  consults  are  the  primary  departments  of  the  CAMO. 
The  internal  consult  department  focuses  on  primary  care  appointments  and  referrals 
within  the  beneficiary’s  enrolled  military  treatment  facility.  The  department  also  has  the 
capability  of  scheduling  consults  at  other  military  treatment  facilities  within  the  SAMSM. 
Agents  within  the  internal  consult  department  have  been  further  specialized  to  increase 
familiarization  with  service-specific  and  clinic-specific  standard  operating  procedures. 
Currently,  the  CAMO  serves  eight  primary  care  clinics  from  Wilford  Hall  Medical 
Center,  six  primary  care  clinics  from  Brooke  Army  Medical  Center,  and  four  primary 
care  clinics  from  Randolph  Air  Force  Base.  Keeping  appointments  within  the  direct  care 
system  saves  resources,  maintains  continuity  of  care,  and  improves  graduate  medical 
education  opportunities  (S.  Graham,  personal  communication,  May  25,  2006). 

Appendix  B  illustrates  the  process  for  internal  consults.  All  internal  consults  are 
required  to  be  entered  electronically  in  AHLTA.  The  CAMO  routes  the  referral  to  the 
appropriate  team  or  agent  based  on  specialization.  Appointing  within  the  beneficiary’s 
enrolled  treatment  facility  is  the  first  priority.  If  the  enrolled  treatment  facility  is  not 
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available,  the  CAMO  attempts  to  appoint  based  on  geographic  proximity,  patient 
preference,  and  service  affiliation.  Patient  appointing  follows  clinic-specific  standard 
operating  procedures.  All  clinics  have  the  right  to  review  consults  before  accepting. 

Once  accepted  by  the  clinic,  the  CAMO  contacts  the  patient  to  schedule  the  appointment. 

If  access  standards  can  not  be  achieved  within  the  direct  care  system,  the  consult 
is  routed  to  the  external  consult  department.  All  appointments  and  consults  that  are 
deferred  to  the  network  are  managed  by  the  external  consult  department.  Following 
medical  management  review,  consults  are  forwarded  to  the  managed  care  support 
contractor.  The  CAMO  does  not  make  appointments  with  network  providers.  The 
external  consult  department  also  manages  the  right  of  first  refusal  process  and  retrieval  of 
results  from  network  providers.  Staff  members  are  cross-trained  to  augment  other 
departments  during  peak  call  volume  periods. 

The  CAMO  plays  a  critical  role  in  the  right  of  first  refusal  (ROFR)  process  within 
the  SAMSM.  The  right  of  first  refusal  process  is  one  management  tool  that  supports  the 
optimal  utilization  of  the  direct  care  system.  The  process  requires  the  managed  care 
support  contractor  to  notify  the  direct  care  system  of  additional  treatment  needs  of 
beneficiaries  referred  to  the  network.  An  enrolled  beneficiary  may  only  be  deferred  to 
the  TRICARE  network  if  a  military  treatment  facility  lacks  the  capacity  to  provide  the 
standard  of  care  (Health  Affairs,  2005).  Current  TRICARE  contracts  provide  incentives 
for  contractors  and  the  military  healthcare  system  to  maximize  the  use  of  the  direct  care 
system. 

Aggressive  management  of  the  right  of  first  refusal  process  in  the  SAMSM  has 
led  to  an  average  acceptance  rate  of  78  percent.  Military  treatment  facilities  are  required 
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to  provide  the  managed  care  support  contractor  with  accurate  listings  of  clinics  with 
capacity  and  capabilities  to  accept  referrals  from  the  network  (Health  Affairs,  2005).  Up 
to  date  and  accurate  listings  expedite  the  referral  process.  Military  treatment  facilities 
can  eliminate  unnecessary  wait  times  for  beneficiaries  by  not  listing  services  that  do  not 
have  capacity  to  receive  external  referrals.  The  SAMSM  monitors  clinic  utilization  and 
right  of  first  refusal  rates  on  a  monthly  basis.  Adjustments  to  the  availability  listing  are 
updated  as  needed  to  maximize  the  use  of  the  direct  care  system  and  minimize 
unnecessary  administrative  procedures  for  beneficiaries. 

The  CAMO  provides  the  central  node  to  coordinate  available  resources.  Clinics 
review  all  consults  to  determine  whether  or  not  capacity  exists  to  accept  a  referral. 
Decisions  are  made  by  clinical  staff,  not  agents  of  the  CAMO,  as  to  the  acceptance  or 
deferral  of  a  consult.  Appendix  C  illustrates  the  process  for  the  right  of  first  refusal. 

Further,  the  CAMO  provides  a  single  source  for  military  and  network  providers  to 
report  results  of  consults.  This  facilitates  continuity  of  care  and  maintenance  of  the 
beneficiary’s  electronic  health  record.  Timely  information  exchange  between  primary 
care  physicians  and  specialists  is  critical  to  the  continuity  of  care  in  a  referral  process 
(Murray,  2002).  The  Office  of  the  Assistant  Secretary  of  Defense,  Health  Affairs  issued 
standards  for  the  referral  process  to  support  the  standard  of  care.  The  right  of  first  refusal 
policy  requires  the  military  treatment  facility  to  provide  clinical  feedback  to  the  referring 
civilian  provider  within  10  business  days  of  the  appointment.  The  standards  also  require 
the  military  treatment  facility  to  accept  or  refuse  the  referral  within  24-hours.  The 
response  to  the  managed  care  support  contractor  should  be  as  soon  as  possible,  but  not 
more  than  one  business  day  (Health  Affairs,  2005).  The  CAMO  coordinates  the 
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reporting  process  and  provides  administrative  oversight  to  ensure  results  of  consults  are 
completed  by  network  and  military  providers. 

Appointment  and  consult  protocols  are  clearly  defined  by  the  management  of  the 
CAMO.  However,  clinics  have  maintained  autonomy  with  respect  to  standard  operating 
procedures  for  appointing.  The  business  rules  of  the  CAMO  must  capture  the 
individualized  nature  of  the  1 00  clinics  within  the  SAMSM.  Some  best  business 
practices  are  being  adopted  to  standardize  the  appointing  process,  yet  the  nature  of 
healthcare  requires  a  certain  level  of  customization  at  the  clinic  level  (S.  Graham, 
personal  communication.  May  25,  2006). 

Agents  of  the  CAMO  have  access  to  AHLTA.  All  appointments  are  scheduled 
directly  into  the  same  electronic  system  used  by  the  clinics.  Entries  by  the  CAMO 
become  a  part  of  the  beneficiary’s  health  record.  Appointment  templates  in  AHLTA 
must  be  released  by  the  individual  clinics  in  order  to  be  available  for  CAMO  agents  to 
schedule.  Initially,  this  presented  a  problem  because  templates  were  not  available. 

Industry  Trends  for  Appointing  and  Referral  Management 

Data  for  appropriate  measures  must  be  collected  and  analyzed  to  determine  the 
positive  or  negative  impact  of  changes  to  the  appointment  and  referral  process.  A  post¬ 
implementation  decision  audit  should  consider  the  waiting  time  between  referral  and 
appointment  with  specialist  (Murray,  2002).  Analysts  for  the  SAMSM  currently  track 
two  separate  metrics  to  monitor  this  information.  The  time  from  the  initiation  of  the 
referral  until  an  appointment  is  scheduled  is  the  first  metric.  The  wait  time  from  the 
initiation  of  the  referral  until  the  actual  appointment  with  the  specialist  is  the  second 


CAMO  Analysis  1 5 


metric.  The  TRICARE  access  standards  state  that  a  specialty  appointment  must  be 
available  within  28  days  of  the  referral.  The  data  provide  a  measure  for  the  adequacy  of 
access  within  the  network.  However,  the  integrity  of  data  prior  to  2004  remains  suspect, 
as  it  was  not  specifically  monitored  or  reported  on  a  regular  basis. 

Patient  satisfaction  provides  a  key  measure  of  the  success  of  any  health  care 
initiative.  Murray  (2002)  stresses  the  importance  of  evaluating  changes  in  patient 
satisfaction  associated  with  involvement  in  the  appointment  and  referral  process. 
Aggregate  patient  satisfaction  data  are  available  through  the  TRICARE  Customer 
Satisfaction  Survey  (CSS).  The  National  Research  Corporation  administers  the  Customer 
Satisfaction  Survey  on  a  weekly  basis  to  patients  seen  in  clinics  within  the  continental 
United  States  and  overseas.  Patients  rate  the  quality  of  the  healthcare  encounter  using  1 7 
multiple  choice  questions.  A  copy  of  the  survey  is  included  in  Appendix  D. 

The  appointing  system  for  the  military  healthcare  system  has  been  scrutinized  for 
its  connection  to  patient  satisfaction.  The  United  States  General  Accountability  Office 
(2000)  conducted  an  inquiry  entitled  Military  Health  Care:  Factors  Affecting  Contractors' 
Ability  to  Schedule  Appointments.  The  report  cited  confusing  appointment  procedures 
which  led  to  inefficiencies  and  failure  to  provide  access  within  the  standard  for  patient 
care.  Confusion  and  frustration  continue  to  be  a  problem  when  beneficiaries  attempt  to 
schedule  appointments.  In  an  article  published  in  Military  Medicine,  Colonel  Bonnie 
Jennings  (2005)  discussed  the  appointment  obstacle  course  that  beneficiaries  must 
negotiate  to  obtain  an  appointment.  A  common  theme  was  reported  that  referenced  an 
unproductive  cycle  of  calling  and  recalling  for  appointments.  Beneficiaries  were  being 
told  that  all  appointments  were  full  and  that  the  new  schedule  of  appointments  was  not 
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available.  When  the  beneficiary  called  back  at  the  recommended  time,  the  new 
appointment  schedule  was  already  full. 

Administrators  monitor  appointment  templates  and  attempt  to  optimize  the  mix  of 
appointment  types.  Fill  rates  provide  a  strong  indicator  of  how  the  appointment 
templates  are  meeting  the  demand  of  the  beneficiary  population.  To  determine  the  fill 
rate,  the  scheduled  appointments  are  divided  by  the  total  number  of  appointments 
available.  The  ideal  fill  rate  is  between  90  percent  and  95  percent  (Buttz,  2004). 
Maintaining  the  appointment  template  six  weeks  into  the  future  has  posed  a  problem  for 
many  clinics.  Managing  the  availability  of  a  template  with  the  optimal  mix  can  alleviate 
patient  frustrations. 

The  medical  industry  has  been  attempting  to  define  best  business  practices  in 
patient  appointing.  According  to  the  research  of  Kent  Seltman  (2005),  “There  is  no  best 
practice  in  appointment  management.”  Seltman  works  at  the  Mayo  Clinic  in  Rochester, 
Minnesota  as  the  director  of  marketing.  In  his  article  in  Marketing  Health  Services, 
Seltman  cited  four  problem  areas  with  patient  appointing  system.  The  problems  involved 
triage  criteria,  appointing  staff,  customer  service,  and  a  bias  to  say  no.  The  triage  issue 
relates  to  the  fact  that  reviewing  the  records  of  referred  patients  takes  time.  Reviewing 
referrals  is  not  billable.  Providers  stay  busy  with  revenue-generating  workload.  A 
patient  who  is  forced  to  wait  may  seek  care  elsewhere,  based  on  medical  need  or 
impatience.  The  end  result  is  a  lost  appointment.  Appointment  staffing  refers  to  the 
competency  of  the  agents  designated  to  schedule  appointments.  Appointment  clerks  are 
often  low-pay,  entry-level  positions.  Levels  of  training  in  medical  terminology  vary 
greatly.  Staff  turnover  is  a  common  problem  among  entry-level  positions,  as  well. 
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Customer  service  is  an  extension  of  staffing  decisions  and  training.  Seltman’s  research 
indicated  problems  with  simple  details  of  customer  service  including  common  courtesies, 
conversational  tone,  and  number  of  rings  before  a  call  is  answered.  The  bias  to  say  no 
stems  from  a  fear  of  making  a  mistake.  Instead  of  scheduling  an  incorrect  appointment, 
clerks  choose  not  to  make  the  appointment.  Training  and  supervision  of  the  appointing 
staff  can  improve  three  of  the  four  problem  areas  (Seltman,  2005). 

The  TRICARE  Management  Activity  publishes  the  telephonic  access  standards 
and  benchmarks  for  the  military  healthcare  system.  Telephonic  and  internet  based 
customer  service  standards  have  become  increasingly  competitive  in  the  private  sector. 
Healthcare  systems  for  customer  service  generally  lag  the  private  sector  (Seltman,  2005). 
The  military  healthcare  system,  however,  established  a  benchmark  to  equal  the  corporate 
standard  for  answering  a  telephone,  which  is  within  four  rings  or  30  seconds  (TRICARE 
Management  Activity,  n.d.).  Hold  time  is  defined  as  the  time  after  the  call  is  answered 
until  the  caller  speaks  to  an  appointment  scheduler.  The  TRICARE  benchmark  for  hold 
time  is  two  minutes.  The  abandoned  call  rate  is  the  percentage  of  calls  that  hang  up  after 
the  benchmark  for  answering  the  telephone.  The  current  benchmark  for  abandoned  call 
rate  is  five  percent  or  less.  The  final  benchmark  is  process  time  of  less  than  six  minutes 
for  large  treatment  facilities.  Process  time  is  measured  by  the  time  it  takes  a  scheduler  to 
make  the  appointment  and  complete  the  call.  The  TRICARE  Management  Activity 
maintains  benchmarks  as  indicators  of  efficiency,  training,  and  required  staffing  levels 
for  appointment  schedulers  (TRICARE  Management  Activity,  n.d.). 

The  complexity  of  referrals  and  consults  requires  full-time  management  for  the 
process  even  in  small  family  practices.  John  Spicer  (1998),  associate  editor  for  Family 
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Practice  Management,  recommended  referral  management  by  dedicated  staff  members 
trained  in  the  details  of  the  process.  He  also  recommended  two  suggestions  for 
improving  referral  management.  Standardization  of  the  referral  requirements,  forms,  and 
protocol  was  the  first  recommendation.  Developing  shortcuts  or  matrices  to  navigate  the 
process  was  the  second  recommendation.  The  CAMO  accomplishes  both  of  Spicer’s 
recommendations.  While  continuing  to  push  for  standardization  among  the  clinics,  the 
CAMO  has  heightened  visibility  of  best  practices  and  potential  problem  areas.  The 
CAMO  established  a  format  for  standard  operating  procedures  that  include  a  matrix  for 
all  appointment  types  and  an  algorithm  format  to  assist  in  matching  patients  to  the 
appropriate  clinic. 

Current  trends  in  the  Department  of  Defense  and  the  military  healthcare  system 
necessitate  heightened  accountability  for  all  types  of  resources.  The  global  war  on  terror, 
soaring  medical  inflation,  and  increased  medical  benefits  to  service  members  have  lead  to 
a  number  of  changes  in  the  military  healthcare  system.  In  2005,  the  Surgeon  General  of 
the  Army,  Lieutenant  General  Kevin  Kiley,  directed  increased  emphasis  on  workload 
accountability,  coding  accuracy,  and  data  quality.  In  several  memorandums  to  the 
Regional  Medical  Commands,  Lieutenant  General  Kiley  outlined  his  vision  and 
expectations  for  CHCS  II  implementation  (Army  Medical  Command,  2005,  January).  He 
issued  specific  implementation  guidance  for  the  enterprise  plan  to  improve  coding  (Army 
Medical  Command,  2005,  June).  The  emphasis  was  tied  to  the  transition  of  the  Army 
Medical  Department  from  the  traditional  form  of  budgeting  to  a  prospective  payment 


system. 
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The  movement  to  the  electronic  health  record  has  influenced  significant  changes 
in  data  collection  and  managerial  behavior.  Improved  data  quality  combined  with  the 
proliferation  of  information  management  systems  to  collect  and  process  data  provide 
managers  the  opportunity  to  make  sound  business  decisions.  However,  the  improvement 
of  data  quality  limits  this  study’s  ability  to  make  accurate  comparisons  between  pre¬ 
implementation  and  post-implementation  performance. 

Purpose 

The  purpose  of  this  analysis  was  to  develop  a  framework  to  evaluate  the  impact  of 
the  Consult  and  Appointment  Management  Office  (CAMO)  on  the  delivery  of  healthcare 
in  the  San  Antonio  Multi-Service  Market. 

Methods  and  Procedures 

Research  Design 

The  research  design  of  this  analysis  was  based  on  an  exploratory  case  study.  As 
recommended  by  Yin  (1989),  the  composition  of  the  study  followed  a  linear-analytic 
structure.  The  general  outline  of  this  case  included  a  statement  of  the  problem, 
explanation  of  the  analytical  methods  employed,  findings  of  the  analysis,  and  conclusions 
and  recommendations.  Case  Study  Research:  Design  and  Methods  by  Robert  K.  Yin 
provided  the  primary  reference  for  this  study’s  methods  and  procedures. 

Three  key  factors  contributed  to  the  selection  of  an  exploratory  case  study  as  the 
appropriate  format  for  this  analysis.  First,  the  CAMO  is  a  contemporary  issue  for  which 
limited  information  exists,  especially  within  the  Military  Healthcare  System.  The  San 


CAMO  Analysis  20 


Antonio  Multi  Service  Market  was  the  first  and  only  organization  to  centrally  manage 
consults,  referrals,  and  appointing  of  this  scale  in  the  Department  of  Defense.  Second, 
the  lack  of  concrete  boundaries  and  contextual  relationships  posed  a  major  challenge  for 
this  analysis.  Finally,  the  variables,  to  be  defined  in  the  following  section,  were  not  able 
to  be  manipulated  for  this  study.  Although  additional  justification  exists,  Yin  made 
specific  reference  to  these  factors  in  his  definition  of  an  exploratory  case  study. 

Sources  of  Information 

Multiple  sources  of  data  were  used  for  this  study.  Yin  (1989)  outlined  six  sources 
of  evidence  and  recommended  using  all  relevant  sources  available  to  increase  content 
validity.  The  six  sources  include:  documentation,  archived  records,  interviews,  direct 
observations,  participation-observation,  and  physical  artifacts.  Physical  artifacts  were  not 
used  for  this  analysis. 

This  study  included  an  extensive  review  of  documentation.  The  literature  review 
involved  Ovid,  MedLine,  and  Google  searches  for  journals  articles,  news  clippings,  and 
similar  studies.  The  TRICARE  Management  Activity  provided  the  primary  source  for 
policy  guidelines,  and  standards.  Local  memorandums,  reports,  and  presentations  by  the 
CAMO  and  SAMSM  provided  detailed  information  at  various  levels  of  operation. 
Documentation  provided  the  largest  volume  of  background  information. 

Archived  records  generated  the  majority  of  empirical  data  for  analysis.  Data  from 
the  following  computer  systems  were  employed:  Composite  Health  Care  System 
(CHCS),  the  M2  Information  Data  Base,  and  the  local  Automated  Call  Distribution 
Systems.  Patient  satisfaction  data  were  pulled  from  the  Customer  Satisfaction  Survey 
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database  maintained  by  Dr.  Mangelsdorff.  The  database  provided  patient  satisfaction 
scores  on  a  monthly  basis  for  all  clinics  in  the  SAMSM.  The  Automated  Call 
Distribution  System  provided  historical  and  current  average  patient  hold  times,  call 
abandonment  rates,  and  call  volumes.  CHCS  and  M2  provided  data  on  consult  reporting 
and  administrative  closure  rates.  Population  and  sample  demographics  were  also  pulled 
from  M2. 

Interviews  were  conducted  with  key  personnel  of  the  CAMO,  Department  of 
Health  Care  Operations,  and  TRICARE  Regional  Office.  The  interviews  involved  open- 
end  questions  concerning  experiences,  challenges,  successes,  opinions,  and 
recommendations.  The  primary  purpose  of  the  interviews  was  to  gain  general 
information  and  understanding  about  the  CAMO. 

Direct  observation  was  also  used  for  this  study.  The  CAMO’s  current  phone 
system  enables  an  authorized  third  party  to  monitor  calls.  The  author  spent  one  week 
monitoring  calls  in  each  department  of  the  CAMO.  This  established  a  firm  understanding 
of  the  standard  operating  procedures,  as  well  as  the  challenges  faced  by  operators 
required  to  navigate  the  inconsistent  and  complex  appointment  templates.  The  author 
also  employed  participant  observation  while  making  three  personal  appointments  during 
the  study. 

Methodology 

The  Ten  Steps  for  Hypothesis  Testing  -  Functional  Form  Statistics  (Finstuen, 
2004)  will  provide  the  format  to  analyze  the  statistical  significance  of  relationships 
between  the  variables.  Data  will  be  entered  into  the  Statistical  Package  for  the  Social 
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Sciences  (SPSS)  software  to  compute  descriptive  and  inferential  statistics.  Graphic 
summaries,  correlations,  trend  analysis,  and  analysis  of  variance  will  be  used  to  evaluate 
the  impact  of  the  CAMO. 

The  unit  of  analysis  is  defined  by  three  characteristics  including:  social  context, 
time,  and  space.  The  social  context  for  this  study  will  focus  on  data  at  the  clinic  level. 
Two  twelve-month  time  periods  will  be  used.  Fiscal  year  2004  will  provide  the  baseline 
for  measurements  before  the  implementation  of  the  CAMO.  Data  for  post¬ 
implementation  will  be  based  on  performance  from  April  2005  through  March  2006.  The 
second  time  period  represents  the  most  current  and  complete  data  available  for  the  study. 
Both  time  periods  will  reflect  the  natural  seasonality  that  occurs  over  a  12-month  period. 
The  space  will  include  all  clinics  within  the  SAMSM. 

The  CAMO  serves  as  a  tool  to  optimize  use  of  the  direct  care  system  in  the 
SAMSM.  Therefore  the  population  of  this  study  includes  all  clinics  within  the  SAMSM 
from  October  2003  through  March  2006.  The  sample  will  include  all  clinics  within  the 
SAMSM  with  adequate  levels  of  data  for  selected  variables  during  both  time  periods  to 
support  statistical  analysis. 

Operational  Definitions  of  Variables 

This  study  will  examine  aspects  of  cost,  quality,  and  access  to  determine  the 
impact  of  the  CAMO  on  healthcare  delivery  in  the  San  Antonio  Multi  Service  Market. 
Figure  l  represents  the  proposed  model  for  this  study. 


CAMO  Analysis  23 


Healthcare  Delivery 


Figure  1:  Conceptual  model  of  case  study 

The  analysis  of  costs  will  focus  on  payroll  and  personnel  efficiencies.  The  one¬ 
time  setup  costs  will  also  be  investigated.  Determinations  related  to  quality  will  be  based 
on  patient  satisfaction  and  referral  management.  Patient  satisfaction  indicators  will  be 
pulled  from  the  Military  Healthcare  System  Monthly  Customer  Satisfaction  Survey. 
Referral  management  measures  from  M2  and  CHCS  will  indicate  continuity  of  care 
between  primary  and  specialty  providers.  Rates  for  resulted  consults  and 
administratively  closed  consults  will  be  analyzed.  Access  to  the  healthcare  system  will 
focus  on  telephonic  access  to  schedule  appointments.  A  complete  listing  of  the 
operational  definitions  for  measures  and  variables  are  included  in  Appendix  E.  Although 
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data  are  available  on  a  daily  basis  for  some  variables,  this  study  will  use  monthly 
averages,  unless  otherwise  stated,  to  standardize  the  analysis. 

Variables  for  this  study  have  been  chosen  based  on  the  literature  and 
recommendations  of  key  management  personnel  from  the  CAMO.  Certain  relationships 
concerning  the  variables  have  been  proposed.  This  study  will  attempt  to  validate  these 
relationships  in  order  to  determine  the  impact  of  the  CAMO  on  the  delivery  of  healthcare 
in  the  SAMSM. 

The  first  hypothesis  (Hai)  states  the  CAMO  improved  access  to  healthcare  by 
decreasing  average  telephone  hold  times  for  patients  attempting  to  schedule 
appointments.  A  null  hypothesis  (Hoi)  will  be  tested  to  show  that  average  hold  times  did 
not  change  after  implementation  of  the  CAMO.  The  second  hypothesis  (Ha2)  states  the 
CAMO  improved  quality  of  healthcare  by  decreasing  the  rate  consults  being 
administratively  closed.  This  study  will  test  a  null  hypothesis  (H02)  stating  the  rate 
consults  being  administratively  closed  did  not  change  after  implementation  of  the 
CAMO.  The  last  hypothesis  (Haj)  states  the  CAMO  improved  patient  satisfaction  scores. 
The  null  hypothesis  (H03)  states  that  patient  satisfaction  did  not  change  after 
implementation  of  the  CAMO. 

To  evaluate  the  validity  of  the  null  hypothesis  for  each  of  the  independent 
variables,  the  significance  must  be  more  than  the  set  decision  level.  For  all  of  the 
independent  variables  the  decision  level  was  established  at  a  =  .05.  Significance  for  any 
of  the  null  hypotheses  measuring  more  than  .05  suggests  that  the  relationship  occurs  by 
chance,  rather  than  influences  of  the  independent  variable.  The  null  hypotheses  can  only 
be  rejected  if  the  significance  is  .05  or  less. 


CAMO  Analysis  25 


Reliability  and  Validity  Concepts 

Reliability  and  validity  describe  the  quality  of  measurements.  In  this  analysis,  the 
primary  sources  of  data  come  from  professionally  managed  databases  and  automated 
systems.  The  automated  systems  have  specifically  defined  operational  definitions  of  the 
data.  A  key  assumption  in  this  analysis  is  that  the  data  in  the  automated  systems  were 
entered  accurately  and  provide  a  true  representation  of  reality.  Consulting  with  the 
subject  matter  experts  from  the  CAMO  and  SAMSM  helped  to  establish  construct 
validity.  The  operational  definitions  in  this  analysis  are  consistent  with  the  definitions 
used  for  routine  monitoring  and  evaluation  of  the  CAMO. 

Assumptions  and  limitations 

As  stated  in  the  literature  review,  the  military  healthcare  environment  has 
experienced  significant  changes  since  2003.  A  number  of  factors  have  influenced 
changes  in  policy,  corporate  culture,  and  managerial  behavior.  The  factors  include: 

1 .  A  transition  to  a  new  generation  of  TRICARE  contracts 

2.  Unprecedented  command  emphasis  on  workload  accountability,  data  quality, 
and  coding  accuracy 

3.  Implementation  of  AHLTA  and  the  electronic  health  record 

4.  Proliferation  of  information  technology  and  management  systems 

5.  Advances  in  automated  call  distribution  systems 

6.  Budgetary  constraints  associated  with  the  global  war  on  terror  and  the 


escalation  of  medical  inflation 
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7.  Expansion  of  healthcare  benefits  packages  to  Department  of  Defense 
beneficiaries 

In  such  a  dynamic  environment,  isolation  and  measurement  of  a  specific 
components  present  serious  challenges.  Some  of  the  factors  have  positive  influences  on 
productivity,  efficiency,  and  patient  satisfaction.  Other  factors  may  create  a  negative 
effect.  Pre-implementation  and  post-implementation  data  do  not  necessarily  represent 
like  comparisons.  Although  the  intent  of  this  analysis  was  to  produce  a  clean, 
quantitative  evaluation  of  the  impact  of  the  CAMO,  the  results  will  require  a  qualitative 
interpretation. 

One  significant  limitation  for  this  study  involves  data  quality  and  availability  for 
measures  prior  to  the  implementation  of  the  CAMO.  Telephonic  wait  times  were 
recorded,  but  rarely  reported  to  senior  management.  Previous  automated  call  distribution 
systems  lacked  the  capability  for  managerial  oversight.  Incremental  upgrades  to  the 
automated  call  distribution  systems  and  organizational  structure  of  the  CAMO  created 
incremental  impacts  that  this  analysis  does  not  capture. 

The  previous  referral  process  poses  an  additional  challenge.  Prior  to  the 
implementation  of  the  CAMO  and  the  electronic  health  record,  referrals  were  often 
entered  into  the  paper  record  and  electronically  into  the  Composite  Health  Care  System 
(CHCS).  However,  providers  often  entered  the  results  of  referrals  only  in  the  paper 
record.  The  electronic  record  was  not  always  updated.  Therefore,  the  electronically 
entered  referral  would  reflect  an  administrative  closure  after  90  days,  even  though  the 
standard  of  care  was  delivered  to  the  patient.  The  implementation  of  the  electronic  health 
record  eliminated  the  redundancy  of  referral  entry  since  all  entries  and  results  are 
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recorded  electronically.  The  data  available  for  analysis  are  from  the  electronically 
entered  referrals.  The  rate  of  administrative  closures  will  be  higher  in  the  pre¬ 
implementation  period  due  to  the  disparity  between  referral  entry  and  resulting. 

Minor  variations  in  productivity  due  to  specific  events  have  been  identified  within 
the  period  of  analysis.  The  implementation  of  AHLTA  is  the  most  significant  factor. 
During  implementation,  the  number  of  available  appointments  decreased  to  accommodate 
staff  training.  The  benefits  gained  in  coding  accuracy  may  influence  changes  in  the  data, 
although  the  standard  of  care  and  provider  productivity  remain  constant.  The  cumulative 
effect  could  produce  variations  in  patient  satisfaction.  Since  patient  satisfaction  scores 
tend  to  be  consistent  for  the  overall  episode  of  care,  isolating  satisfaction  with  hold  times 
and  access  may  be  skewed. 


Results 

This  analysis  provided  a  framework  to  evaluate  the  impact  of  the  Consult  and 
Appointment  Management  Office  (CAMO)  on  the  delivery  of  healthcare  in  the  San 
Antonio  Multi-Service  Market.  It  operationally  defined  variables  and  measures  used  to 
further  evaluate  performance.  The  methodology  may  be  tailored  for  a  post¬ 
implementation  analysis  of  a  similar  project  or  service  line. 

The  findings  and  results  focused  on  three  factors  of  the  healthcare  delivery 
system,  commonly  referred  to  as  the  iron  triangle  of  healthcare.  Shi  and  Singh  (2004) 
described  the  iron  triangle  as  an  evaluation  framework  to  measure  effectiveness, 
efficiency,  and  equity.  The  three  factors  of  the  iron  triangle  are  cost  of  service  provided. 
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quality  of  care,  and  access  to  care.  While  the  findings  focused  on  quantitative  results, 
qualitative  interpretations  are  also  presented.  Certain  aspects  of  healthcare  delivery  can 
be  better  addressed  qualitatively. 

Cost  of  Service  Provided 

Analysis  of  data  from  the  Medical  Expense  and  Personnel  Reporting  System 
(MEPRS)  and  the  Standard  Finance  System  (STANFINS)  identified  two  general 
categories  of  relevant  costs  associated  with  the  CAMO.  The  categories  included  set  up 
costs  and  costs  for  sustained  operations.  The  one-time  set  up  costs  to  establish  the 
CAMO  involved  renovations  to  BAMC,  initial  outfitting  of  furniture  and  equipment,  and 
upgrades  to  the  automated  call  distribution  system.  Personnel  wages  and  contracts 
comprised  the  greatest  portion  of  cost  for  sustained  operations. 

The  initial  agreement  between  BAMC  and  WHMC  to  establish  the  CAMO 
included  provisions  for  cost  share.  The  CAMO  was  defined  as  a  Multi-Service  Market 
asset,  servicing  the  clinics  and  beneficiary  population  of  BAMC  and  WHMC.  The 
agreement  provided  that  WHMC  would  shoulder  60  percent  of  the  costs  associated  with 
the  CAMO,  leaving  40  percent  for  BAMC  to  pay. 

Costs  of  the  CAMO  were  isolated  in  MEPRS  under  the  four-digit  Functional  Cost 
Code  for  the  Patient  Appointing  System.  The  four-digit  code  for  Patient  Appointing 
System  at  BAMC  is  EKAB.  The  Patient  Appointing  System  falls  under  Ambulatory 
Care  Administration  in  MEPRS. 

The  set  up  costs  represent  the  one-time  costs  associated  with  establishing  the 
CAMO.  Table  1  illustrates  the  total  set  up  costs  for  the  CAMO,  as  well  as  the  portion 
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paid  by  BAMC.  The  Job  Order  Contractor  Proposal  (Appendix  E)  submitted  by  J&J 
Maintenance,  Inc  outlines  the  renovation  costs  for  the  CAMO.  The  initial  outfitting  of 
furniture  included  state  of  the  art  sound-deadening  workstations  with  white  noise 
generation  for  the  operators.  Although  the  Automated  Call  Distribution  System  replaced 
BAMC’s  preexisting  system,  the  initial  cost  was  still  split  between  BAMC  and  Wilford 
Hall.  The  initial  supplies  included  general  office  supplies  and  operator  manuals. 

Table  1 

Summary  of  Set  Up  Costs 


Total  Cost 

BAMC’s  Share 

Renovation 

$  726,372.07 

$ 

290,548.83 

Furniture 

$  576,840.00 

$ 

230,736.00 

Automated  Call  Distribution  System 

$  560,000.00 

$ 

224,000.00 

Initial  Supplies 

$  21,437.71 

$ 

8,575.08 

$  1,884,649.78 

$ 

753,859.91 

Initial  staffing  for  the  CAMO  fluctuated  over  the  first  12  months  of  operation. 
Staffing  stabilized  at  63  full  time  equivalents  (FTE).  Six  part  time  employees  to 
accommodate  peak  calling  times  were  included  in  the  63  FTEs.  The  personnel  costs,  as 
other  costs,  were  split  between  BAMC  and  WHMC.  To  maintain  the  proportion  of 
expenses,  management  decided  to  staff  the  CAMO  with  40  percent  of  the  employees  paid 
for  by  the  Army  and  60  percent  of  the  employees  paid  for  by  the  Air  Force.  The  only 
military  staff  member  working  in  the  CAMO  was  a  Captain  from  the  Air  Force. 
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Staffing  included  a  mix  of  government  civilians  and  contractors.  BAMC 
primarily  used  General  Schedule  (GS)  employees  to  staff  the  CAMO.  Only  two 
contractors  were  hired  by  BAMC.  The  16  GS  employees  from  the  previously  existing 
Patient  Appointing  Service  transitioned  to  fill  positions  in  the  CAMO  along  with  three 
additional  GS  employees.  Table  2  below  shows  the  costs  of  staffing  the  CAMO  for  fiscal 
year  2005. 


Table  2 

CAMO  Staffing  Costs  for  Fiscal  Year  2005 


BAMC 

WHMC 

Total 

GS  Employee  Compensation 

$  736,049 

$  353,219 

$  1,089,268 

Contract  Personnel  Compensation 

$  21,059 

$  846,024 

$  867,083 

Total  Compensation 

S  757,108 

$  1,199,243 

$  1,956,351 

Percentage 

39% 

61% 

100% 

Note:  BAMC  contract  personnel  were  only  utilized  during  the  last  portion  of  the  fiscal 
year. 


The  Air  Force  used  a  larger  ratio  of  contractor  to  GS  employees  to  staff  the 
CAMO.  Twenty-seven  contractors  were  employed  by  the  Air  Force  and  only  10  GS 
employees.  The  flexibility  provided  by  contract  personnel  was  seen  as  a  benefit  during 
the  initial  phases.  Now  that  staffing  levels  have  stabilized  however,  the  Air  Force  is 
considering  converting  the  positions  to  General  Schedule  government  employees. 
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Quality  of  Care  Provided 

This  analysis  evaluated  quality  of  care  based  on  interpersonal  aspects  of  care. 
Patient  satisfaction  surveys  provided  the  interpersonal  aspects  related  to  changes  in 
perceived  quality  of  care.  The  expectation  was  that  interpersonal  aspects  of  quality 
would  improve  with  the  implementation  of  the  CAMO. 

The  Military  Healthcare  System  Monthly  Customer  Satisfaction  Survey  provided 
the  source  for  patient  satisfaction  scores.  Baseline  scores  before  implementation  of  the 
CAMO  were  collected  from  361  encounters  from  fiscal  year  2004.  The  CAMO  was 
established  in  November  2004.  The  post-implementation  evaluation  period  began  in 
January  2005  to  avoid  skewing  results  with  perceptions  related  to  initial  changes  and  the 
associated  learning  curve.  A  total  of  292  encounters  were  collected  during  the  post¬ 
implementation  period.  The  analysis  focused  on  six  questions  in  the  patient  satisfaction 
survey  pertaining  to  perceptions  of  the  appointment  process  and  general  satisfaction. 
Patients  rated  the  following  questions  on  a  7-point  scale. 

Q5:  How  satisfied  were  you  with  the  medical  care  received  at  the  clinic? 

Q6:  Rate  the  number  of  days  between  the  date  that  the  appointment  was  made  and 
the  date  actually  seen  be  provider. 

QlOa:  How  satisfied  were  you  with  the  ease  of  making  the  appointment  by  phone? 

QlOb:  How  satisfied  were  you  with  the  access  to  medical  care  whenever  needed? 

Q1 0c:  How  satisfied  were  you  with  the  process  for  obtaining  referral  for  specialty 
care? 

Q12:  How  satisfied  were  you  with  the  clinic  during  this  visit? 
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Table  3 


Descriptive  Statistics  for  Patient  Satisfactions  Scores 


Question  Number 

Period 

M 

SD 

N 

5 

Pre-Implemtation 

6.15 

1.260 

361 

Post-Implementation 

6.26 

1.239 

292 

6 

Pre-Implemtation 

4.57 

1.721 

361 

Post-Implementation 

4.24 

1.791 

292 

10a 

Pre-Implemtation 

3.99 

1.099 

361 

Post-  Impl  ementation 

3.75 

1.161 

292 

10b 

Pre-Implemtation 

3.95 

1.114 

361 

Post-Implementation 

3.97 

1.033 

292 

10c 

Pre-Implemtation 

4.02 

1.072 

361 

Post-Implementation 

3.97 

1.095 

292 

12 

Pre-Implemtation 

6.01 

1.214 

361 

Post-Implementation 

6.20 

1.110 

292 

The  post-implementation  means  were  compared  to  the  pre-implementation  means 
for  each  question.  Table  3  summarizes  the  means  of  each  question.  Satisfaction 
increased  slightly  for  questions  5,  1  Ob,  and  1 2.  Satisfaction  decreased  for  questions  6, 

1  Oa,  and  1  Oc.  The  analysis  used  a  multivariate  analysis  of  variance  to  compare  the 
difference  in  satisfaction  scores  between  periods.  Table  4  shows  the  statistical 
significance  of  changes  in  means  for  each  question  after  implementation.  Changes  in 
reported  patient  satisfaction  were  statistically  significant  for  questions  6, 10a,  and  12. 
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Table  4 


Multivariate  Analysis  of  Variance  for  Patient  Satisfaction 
between  Periods 


Question  Number 

F 

E 

5 

1.343 

.247 

6* 

5.852 

.016 

10a* 

6.881 

.009 

10b 

.050 

.823 

10c 

.310 

.578 

12* 

4.280 

.039 

Note,  n  =  653  df  =  1  *p  <  .05 


Access  to  Care 

Access  is  measured  in  five  dimensions  including  accessibility,  affordability, 
accommodation,  acceptability,  and  availability.  This  analysis  focused  on  the  temporal 
aspects  of  accessibility.  The  telephonic  hold  times  provided  the  measure  for  evaluation. 
Table  5  below  shows  the  hold  times  and  call  volumes  for  selected  clinics  from  BAMC. 
Clinics  were  selected  based  on  availability  of  pre-implementation  and  post¬ 
implementation  data.  The  only  statistically  significant  change  from  the  pre¬ 
implementation  baseline  occurred  in  the  CAMO.  The  baseline  for  the  CAMO  was  based 
on  the  BAMC  Patient  Appointing  System  telephone  statistics.  The  Patient  Appointing 
System  only  serviced  primary  care  clinics  for  routine  and  follow-up  appointments.  No 
specialty  clinics  or  urgent  care  appointments  were  handled  by  the  Patient  Appointing 
System.  The  increased  average  hold  time  for  the  CAMO  accompanied  the  dramatic 
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growth  in  call  volume.  The  CAMO  expanded  appointing  responsibilities  within  BAMC 
and  assumed  the  appointing  mission  for  WHMC. 


Table  5 


Descriptive  Statistics  for  Telephone  Services  in  Selected  BAMC  Clinics 


Clinic 

Hold  Time  (min:sec) 

Call  Volume  per  Month 

Pre 

Post 

Pre 

Post 

Audiology 

0:15 

0:15 

251 

173 

CAMO 

0:46 

4:40 

19,906 

55,223 

ENT 

0:15 

0:19 

331 

376 

OB/GYN 

1:29 

1:38 

984 

1,198 

Ophthalmology 

1:07 

1:03 

1,053 

740 

Pulmonary 

0:11 

0:12 

228 

195 

Urology 

0:24 

0:36 

546 

597 

Discussion 

The  intent  of  this  project  was  to  use  quantitative  measures  to  evaluate  changes  in 
reported  metrics  from  a  baseline  prior  to  implementation  of  the  CAMO.  Based  on 
dynamic  changes  within  the  Military  Healthcare  System,  quantitative  measures  were 
difficult  to  attribute  solely  to  the  CAMO.  Nonetheless,  the  data  collected  are  valuable  to 
continued  post-implementation  monitoring.  Qualitative  measures  posed  challenges  to 
define,  but  provide  necessary  feedback  when  evaluating  a  business  practice  in  the 
healthcare  industry. 
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Cost  of  Service  Provided 

Isolating  relevant  costs  is  important  in  all  business  decisions.  This  analysis 
focused  on  initial  set  up  costs  and  costs  of  sustained  operations.  However,  not  all  of 
these  costs  are  relevant.  Certain  costs  would  have  been  incurred  regardless  of  the 
CAMO’s  implementation. 

Generally,  all  set  up  costs  are  considered  relevant.  The  total  set  up  cost  for  the 
CAMO  was  $1.9  million.  With  the  cost  share,  BAMC  paid  $753,859  of  the  set  up  costs. 
The  cost  of  renovating  space  in  BAMC  represented  a  total  sunk  cost  of  $726,372.  The 
furniture  and  initial  supplies  were  also  relevant.  From  BAMC’s  perspective  the 
automated  call  distribution  system  does  not  represent  a  relevant  cost,  since  it  replaced  a 
dated  system  and  benefited  the  entire  facility.  The  cost  of  the  new  call  distribution 
system  was  $560,000.  Therefore,  the  relevant  set  up  cost  to  BAMC  equaled  less  than 
$200,000. 

Similar  logic  can  be  applied  to  the  staffing  of  the  CAMO.  Only  differential 
personnel  costs  should  be  considered  in  evaluating  the  cost  of  staffing  the  CAMO.  Since 
BAMC  employed  16  staff  members  in  the  Patient  Appointing  System  prior  to  the 
CAMO,  only  the  costs  of  the  three  new  employees  hired  by  BAMC  are  relevant.  Of  the 
$736,049  BAMC  paid  in  GS  employee  compensation,  the  relevant  cost  equaled  $81,993. 

While  efficiencies  were  gained  in  coordinating  care  within  the  multi-service 
market,  no  reduction  in  staffing  occurred  at  the  clinic  level.  Clinics  retained  receptionists 
for  front  desk  activities  and  managing  patient  calls.  The  call  volumes,  shown  in  Table  5, 
remained  generally  unchanged  after  the  implementation  of  the  CAMO.  Call  volumes  to 
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the  clinics  may  decrease  with  time  as  patient  behavior  changes  to  conform  to  new 
business  practices.  No  long-run  savings  in  personnel  are  expected. 

The  Table  of  Distribution  and  Allowances  (TDA)  for  Brooke  Army  Medical 
Center  has  not  been  updated  to  reflect  the  current  staffing  levels  of  the  CAMO.  Positions 
are  still  held  under  paragraph  number  723,  Patient  Appointing  System.  The  current  TDA 
identifies  the  required  strength  as  1 8  GS  employees,  with  only  2  authorizations.  The 
director  of  the  CAMO  generated  proposed  joint  requirements  and  authorizations 
(Appendix  F).  It  is  unlikely  to  have  a  joint  manning  document  without  a  formal  decision 
on  governance  for  the  San  Antonio  Multi-Service  Market. 

Quality  of  Care  Provided 

Changes  in  patient  satisfaction  were  statistically  significant  for  questions  6,  1  Oa, 
and  12.  Therefore,  the  null  hypothesis  for  patient  satisfaction  (H03)  is  rejected.  However, 
the  alternate  hypothesis  of  improved  patient  satisfaction  does  not  hold  true  either.  Patient 
satisfaction  was  expected  to  improve  for  the  six  areas  of  interest.  Analysis  showed  that 
only  three  areas  improved.  Patients  reported  higher  satisfaction  with  care  at  the  clinic 
(Question  5),  access  to  care  when  needed  (Questions  10b),  and  with  the  clinic  in  general 
(Question  12).  Satisfaction  decreased  for  the  number  of  days  between  making  the 
appointment  and  seeing  a  provider  (Question  6),  ease  of  making  an  appointment  by 
phone  (Question  1 0a),  and  process  for  obtaining  a  referral  for  specialty  care  (Question 
1 0c).  Question  1 0a  which  asked  patients  to  rate  the  satisfaction  with  the  ease  of  making 
the  appointment  by  phone  represented  the  most  significant  decrease  in  satisfaction  (F  = 

6.881,  p<.01). 
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This  analysis  considered  clinical  aspects  for  quality  of  care  in  addition  to 
interpersonal  aspects.  However,  dynamic  changes  within  the  Military  Healthcare  System 
prevented  the  analysis  from  isolating  the  impact  of  the  CAMO  on  changes  to  reported 
metrics.  Adoption  of  the  electronic  health  record,  command  emphasis  on  data  quality, 
and  the  proliferation  of  information  management  software  occurred  simultaneously  with 
the  establishment  of  the  CAMO.  The  changes  in  business  practices  inhibit  the  ability  to 
make  legitimate  comparisons  between  metrics  from  pre-implementation  and  post¬ 
implementation. 

The  most  notable  difficulty  involved  the  analysis  of  administrative  closure  rates 
for  referrals.  In  fiscal  year  2003,  BAMC  began  electronic  entry  of  referrals  and  consults. 
However,  providers  also  generated  hard  copy  referrals  and  consults.  The  consulting 
physician  would  complete  the  record  on  the  hard  copy  medical  record,  but  often  fail  to 
enter  the  results  electronically.  As  a  result,  the  electronic  referral  or  consult  would  be 
administratively  closed  even  though  the  standard  of  care  was  delivered  to  the  patient  and 
documented  on  the  hard  copy  medical  record.  The  Military  Healthcare  System 
implemented  AHLTA  as  the  electronic  medical  record  about  the  same  time  as  the  CAMO 
was  established.  While  the  CAMO  has  made  a  concerted  effort  to  increase  awareness 
and  reduce  the  number  of  referrals  administratively  closed,  changes  in  business  practices 
associated  with  AHLTA  implementation  significantly  reduced  the  artificially  high  rate  of 
referrals  administratively  closed.  Since  a  change  in  administrative  closure  rates  occurred, 
the  null  hypothesis  (H02)  is  rejected.  However,  it  is  not  possible  to  attribute  this  change 
entirely  to  the  implementation  of  the  CAMO. 
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Access  to  Care 

Telephonic  hold  times  provided  the  primary  measure  for  access  in  this  analysis. 
Hold  times  remained  consistent  for  the  clinics,  especially  considering  changes  in  call 
volumes.  Therefore,  the  null  hypothesis  (Hoi)  cannot  be  rejected.  Findings  do  not 
support  the  fact  that  the  CAMO  increased  access  to  healthcare  by  decreasing  average 
telephone  hold  times  for  patients.  The  pre-implementation  and  post-implementation  data 
were  provided  by  the  communication  section  of  BAMC’s  Information  Management 
Division.  No  data  was  available  on  hold  times  for  clinics  from  WHMC. 

Since  inception,  the  director  has  tracked  access  metrics  including  hold  times, 
number  of  appointments  booked,  and  number  of  days  from  consult  ordered  to 
appointment  date.  The  goal  of  the  CAMO  is  to  limit  hold  times  to  less  than  three 
minutes.  The  new  automated  call  distribution  system  provides  additional  monitoring 
capabilities.  Accurately  forecasting  call  volumes  and  scheduling  appropriate  staff  to 
handle  peak  periods  are  essential  to  limiting  call  volumes. 

Analysis  of  survey  results  reveals  that  patient  perceptions  of  access  decreased 
from  baseline  levels.  Satisfaction  in  the  number  of  days  between  when  the  appointment 
was  made  and  when  the  patient  was  seen  by  a  provider  decreased  significantly  ( F  = 

5.852,  p  <  .05).  The  number  of  days  from  consult  to  appointment,  however,  has 
consistently  averaged  25  days.  There  was  no  significant  change  for  satisfaction  with 
access  to  care  when  needed  or  satisfaction  with  the  referral  process. 
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Recommendations 

Management  must  continue  to  monitor  metrics  for  quality  of  care  and  access  to 
care  to  identify  trends.  Changes  to  business  practices  require  an  adequate 
implementation  period  to  allow  time  for  patients  and  staff  to  adapt.  Additional  evaluation 
will  verify  whether  or  not  an  adequate  implementation  period  was  used.  For  instance, 
analysis  of  the  post-implementation  evaluation  period  identified  decreases  in  three  focus 
areas  of  patient  satisfaction.  Decreased  scores  may  have  been  a  reflection  of  the  learning 
curve  for  patients  and  staff  adapting  to  a  new  system.  The  study  design  attempted  to 
account  for  the  learning  curve  by  beginning  the  post-implementation  evaluation  period 
three  months  after  the  CAMO  was  established.  As  more  data  becomes  available,  analysis 
could  confirm  decreased  patient  satisfaction  or  reveal  improved  satisfaction  as  business 
rules  are  solidified  allowing  patients  and  staff  to  become  comfortable  with  the  CAMO. 

The  multi-service  market  poses  significant  challenges  for  administrators.  Policies 
and  service-specific  regulations  impose  additional  constraints.  Any  errors  in  accounting 
or  expense  reporting  result  in  under  or  over  stating  indirect  costs.  To  avoid  erroneously 
reporting  step-down  costs,  careful  consideration  must  be  made  to  identify  the  correct 
accounting  classification  codes  (APC)  and  functional  cost  codes  (FCC).  These  two  codes 
identify  the  activity  generating  the  expense.  When  dealing  with  service  specific  funding, 
it  is  critical  to  track  reimbursable  expenses  and  appropriately  allocate  indirect  costs  to  the 
correct  cost  center. 

Current  staffing  requirements  need  to  be  documented  and  programmed  on  the 
Table  of  Distribution  and  Allowances  (TDA)  for  Brooke  Army  Medical  Center  or  the 
manning  document  for  Wilford  Hall  Medical  Center.  The  deadline  for  the  2009  TDA 
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update  has  already  past.  The  earliest  possibility  for  an  updated  TDA  would  be  the  2010 
version.  The  lack  of  unified  governance  for  the  San  Antonio  Multi-Service  Market 
continues  to  be  the  main  obstacle  to  accomplishing  the  updated  manning  document. 

Conclusion 

The  purpose  of  this  analysis  was  to  develop  a  framework  to  evaluate  the  impact  of 
the  Consult  and  Appointment  Management  Office  (CAMO)  on  the  delivery  of  healthcare 
in  the  San  Antonio  Multi-Service  Market.  The  results  of  this  analysis  provided  insight  to 
the  impact  of  the  CAMO  on  healthcare  within  the  San  Antonio  Multi-Service  Market. 
Extending  the  post-implementation  period  provides  an  opportunity  to  continue  to  monitor 
the  impact  of  the  CAMO.  Additionally,  looking  at  trends  and  performance  since 
implementation  would  provide  additional  support  for  the  merits  of  the  CAMO. 
Consolidation  of  appointing  and  referral  management  poses  many  challenges,  but 
demonstrates  sound  strategic  vision. 
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Appendix  A 

Organizational  Structure  of  the  Consult  and  Appointment  Management  Office 
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Appendix  B 

Internal  Consult  Process  Flowchart 


CAMO  receives 
Consult  and 
pushes  to  Referral 
Clinic  via  CHCS 


Yes 
Yes 

Consult  annotated 
by  Clinic  as  Appt 
toMTF 


CAMO  contacts  pt 
and  books  appt 
Or 

Pt  contacts  CAMO 
to  book 


Unable  to  book 
appt  reason 
determined 


No 


If  unable  to  contact 
Pt  Return  consult 
to  Originating 
Provider  for 
Instruction/Info 
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Appendix  C 

Right  of  First  Refusal  Process  Flowchart 
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Appendix  D 

Operational  Definitions  for  Variables 

Average  hold  time:  The  average  hold  time  is  calculated  by  the  automatic  call  distribution 
system  and  reported  in  minutes.  The  time  begins  once  the  caller  is  placed  into  the 
queue  for  the  next  available  operator.  The  time  ends  either  when  the  caller  is 
connected  to  an  operator  or  the  caller  abandons  the  system. 

Abandonment  rate:  The  abandonment  rate  is  a  percentage  calculated  by  the  number  of 
calls  disconnected  prior  to  being  answered  by  an  operator  divided  by  the  number 
of  calls  that  entered  the  queue. 

Call  volume:  The  call  volume  is  a  simple  count  of  calls  to  a  specific  telephone  number 
or  clinic. 

Patient  Satisfaction  Scores:  The  following  questions  will  be  analyzed  from  the 

TRICARE  Customer  Satisfaction  Survey.  Participants  in  the  survey  respond  to 
questions  on  a  seven-point  bipolar  scale,  with  seven  being  the  most  favorable. 

Q5:  How  satisfied  were  you  with  the  medical  care  received  at  the  clinic? 

Q6:  Rate  the  number  of  days  between  the  date  that  the  appointment  was  made 
and  the  date  actually  seen  be  provider. 

Q 1  Oa:  How  satisfied  were  you  with  the  ease  of  making  the  appointment  by 
phone? 

Q1  Ob:  How  satisfied  were  you  with  the  access  to  medical  care  whenever  needed? 
Q1  Oc:  How  satisfied  were  you  with  the  process  for  obtaining  referral  for 
specialty  care? 
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Appendix  D  (continued) 

Q12:  How  satisfied  were  you  with  the  clinic  during  this  visit. 

Administrative  closure  rate:  The  administrative  closure  rate  is  calculated  by  dividing  the 
total  number  of  consults  from  a  give  clinic  administratively  closed  after  90  days 
due  to  inactivity  by  the  total  number  of  consults  generated  by  the  clinic.  The  rate 
is  reported  as  a  percentage. 

Percentage  of  consults  resulted:  The  percentage  of  consults  resulted  is  calculated  by 

dividing  the  total  number  of  consults  from  a  given  clinic  that  resulted  by  the  total 
number  of  consults  generated  by  a  given  clinic. 

Clinic  uses  CAMO:  This  is  a  dichotomous  variable  measuring  whether  or  not  the  clinic 
allows  the  CAMO  to  schedule  appointments.  Clinics  that  allow  the  CAMO  to 
schedule  appointments  are  coded  1,  0  otherwise.  This  variable  applies  only  to  the 
April  2005-March  2006  time  period. 
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Appendix  E 

BAMC  Renovation  Estimate 


Job  Order  Contractor  Proposal  Cover  Letter 

Date:  09/16/2004 

To:  BAMC  /  US  Army  Corps  Of  Engineers 

From:  J  &  J  Maintenance,  Inc. 

3755  Capital  of  Texas  Hwy  South  Suite  355 
Austin,  TX  78704 


Project  No:  C0114-W1-04 

Project:  Shell  Space  Changes 


Location:  Fort  Sam  Houston,  Texas  BAMC 


SUMMARY  BY  CSI 

CSI  Description 

Installation 

Demolition 

01  General  Conditions 

3,163.20 

000 

02  Site  Work 

18,826.75 

0.00 

03  Concrete 

1,549.45 

0.00 

05  Metals 

1.247.06 

000 

06  Wood  &  Plastic 

3.465.67 

000 

07  Thermal  &  Moisture  Protection 

7.500.48 

0.00 

08  Doors  &  Windows 

36.745.62 

0.00 

09  Finishes 

271.216.93 

000 

10  Specialties 

10.374.50 

3535 

1 1  Equipment 

190.81 

948  69 

1 5  Mechanical 

394,433.09 

9.51 

15  Electrical 

216.981.75 

360  23 

SUMMARY  BY  FACTOR 

Factor  Description 

Installation 

Demolition 

H  1  2700  WEST  CONTRACT  -  J&J  -  FY04 

965.717.36 

1.353  79 

Installation  Total  $965,717.36 

Non  Pre-Priced  % 

0.00% 

Demolition  Total  $1,353.79 

Job  T otal  with  factor  $967,07 1.15 

Job  Total  w/o  Factor 

$761,473.35 

Job  Order  Proposal  Amount 

$967,071.15 

ACF  0.7600  Adjustment 

$-232,097.07 

Discount  (Lump  Sum) 

$-8,602.00 

Proposal  Grand  Total  $726,372.07 


£2 


Contractor^  Signature 

^  *}  fit,  i  jLj 


Date 
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Appendix  F 

Proposed  Manning  Document  for  CAMO 


San  Antonio  -  Consult  &  Appointment  Management  Office 

Section 

Req 

Asgn 

Position/Cont 

Comp 

Notes 

SA-CAMO  Management 

7 

7 

Director 

1 

1 

GS-13 

AF 

Assistant  Director 

1 

1 

GS-11 

Army 

Operations  Officer 

1 

1 

TBD 

AF 

Administrative  Support 

1 

1 

GS-05 

AF 

UR/UM/Ed/Tng  Lead 

2 

2 

GS-06 

Army 

Consult  Routing 

1 

1 

Contract 

AF 

Appointing  Team  A  (CAMO) 

21.5 

21 

Appointing  Supervisor 

1 

1 

GS-07 

AF 

Internal  Consult  Agent  Lead 

1 

1 

GS-06 

Army 

Appointing  Agent 

6 

6 

GS-05 

Army 

Appointing  Agent 

1 

1 

GS-05 

AF 

Appointing  Agent 

9 

8.5 

Contract 

AF 

Appointing  Agent 

0.5 

0.5 

Contract 

AF 

Part  time  0630-1030  M-F 

Appointing  Agent 

1 

1 

Contract 

AF  -  R 

Part  time  0630-1030  M-F 

Appointing  Agent 

2 

2 

Contract 

Army 

Appointing  Team  B  (CAMO) 

21.5 

20.5 

Appointing  Supervisor 

1 

1 

GS-07 

Army 

Internal  Consult  Agent  Lead 

1 

1 

GS-06 

Army 

Appointing  Agent 

3 

3 

GS-05 

Army 

Appointing  Agent 

2 

2 

GS-05 

AF 

Appointing  Agent 

11 

10 

Contract 

AF 

Appointing  Agent 

1.5 

1.5 

Contract 

AF-R 

Part  time  0630-1030  M-F 

Appointing  Agent 

2 

2 

Contract 

Army 

Referral  MGT  (CAMO)  External 

13 

13 

External  Consult  Supervisor 

1 

1 

GS-07 

AF 

External  Consult  Agent  Temp  Lead 

1 

1 

GS-06 

Army 

External  Consult  Agent 

3 

3 

GS-05 

Army 

External  Consult  Agent  (ROFR) 

1 

1 

GS-05 

AF 

External  Consult  Agent 

1 

1 

GS-05 

AF 

External  Consult  Agent  (ROFR) 

1 

1 

Contract 

AF 

External  Consult  Agent 

4 

4 

Contract 

AF 

Appointing  Agent 

1 

1 

Contract 

AF 

Total  pers  Reqd/Auth/Assign 

63 

61.5 

